Neural Network Based Reinforcement Learning for Audio-Visual Gaze Control in Human-Robot Interaction

This paper introduces a novel neural network-based reinforcement learning approach for robot gaze control. Our approach enables a robot to learn and to adapt its gaze control strategy for human-robot interaction neither with the use of external sensors nor with human supervision. The robot learns to focus its attention onto groups of people from its own audio-visual experiences, independently of the number of people, of their positions and of their physical appearances. In particular, we use a recurrent neural network architecture in combination with Q-learning to find an optimal action-selection policy; we pre-train the network using a simulated environment that mimics realistic scenarios that involve speaking/silent participants, thus avoiding the need of tedious sessions of a robot interacting with people. Our experimental evaluation suggests that the proposed method is robust against parameter estimation, i.e. the parameter values yielded by the method do not have a decisive impact on the performance. The best results are obtained when both audio and visual information is jointly used. Experiments with the Nao robot indicate that our framework is a step forward towards the autonomous learning of socially acceptable gaze behavior.Comment: Paper submitted to Pattern Recognition Letter

A Review of Symbolic, Subsymbolic and Hybrid Methods for Sequential Decision Making

The field of Sequential Decision Making (SDM) provides tools for solving Sequential Decision Processes (SDPs), where an agent must make a series of decisions in order to complete a task or achieve a goal. Historically, two competing SDM paradigms have view for supremacy. Automated Planning (AP) proposes to solve SDPs by performing a reasoning process over a model of the world, often represented symbolically. Conversely, Reinforcement Learning (RL) proposes to learn the solution of the SDP from data, without a world model, and represent the learned knowledge subsymbolically. In the spirit of reconciliation, we provide a review of symbolic, subsymbolic and hybrid methods for SDM. We cover both methods for solving SDPs (e.g., AP, RL and techniques that learn to plan) and for learning aspects of their structure (e.g., world models, state invariants and landmarks). To the best of our knowledge, no other review in the field provides the same scope. As an additional contribution, we discuss what properties an ideal method for SDM should exhibit and argue that neurosymbolic AI is the current approach which most closely resembles this ideal method. Finally, we outline several proposals to advance the field of SDM via the integration of symbolic and subsymbolic AI

NeSIG: A Neuro-Symbolic Method for Learning to Generate Planning Problems

In the field of Automated Planning there is often the need for a set of planning problems from a particular domain, e.g., to be used as training data for Machine Learning or as benchmarks in planning competitions. In most cases, these problems are created either by hand or by a domain-specific generator, putting a burden on the human designers. In this paper we propose NeSIG, to the best of our knowledge the first domain-independent method for automatically generating planning problems that are valid, diverse and difficult to solve. We formulate problem generation as a Markov Decision Process and train two generative policies with Deep Reinforcement Learning to generate problems with the desired properties. We conduct experiments on several classical domains, comparing our method with handcrafted domain-specific generators that generate valid and diverse problems but do not optimize difficulty. The results show NeSIG is able to automatically generate valid problems of greater difficulty than the competitor approaches, while maintaining good diversity

Custom Structure Preservation in Face Aging

In this work, we propose a novel architecture for face age editing that can produce structural modifications while maintaining relevant details present in the original image. We disentangle the style and content of the input image and propose a new decoder network that adopts a style-based strategy to combine the style and content representations of the input image while conditioning the output on the target age. We go beyond existing aging methods allowing users to adjust the degree of structure preservation in the input image during inference. To this purpose, we introduce a masking mechanism, the CUstom Structure Preservation module, that distinguishes relevant regions in the input image from those that should be discarded. CUSP requires no additional supervision. Finally, our quantitative and qualitative analysis which include a user study, show that our method outperforms prior art and demonstrates the effectiveness of our strategy regarding image editing and adjustable structure preservation. Code and pretrained models are available at https://github.com/guillermogotre/CUSP.Comment: 36 pages, 21 figure

Employing deep learning for sex estimation of adult individuals using 2D images of the humerus

Biological profile estimation, of which sex estimation is a fundamental first stage, is a really important task in forensic human identification. Although there are a large number of methods that address this problem from different bone structures, mainly using the pelvis and the skull, it has been shown that the humerus presents significant sexual dimorphisms that can be used to estimate sex in their absence. However, these methods are often too subjective or costly, and the development of new methods that avoid these problems is one of the priorities in forensic anthropology research. In this respect, the use of artificial intelligence may allow to automate and reduce the subjectivity of biological profile estimation methods. In fact, artificial intelligence has been successfully applied in sex estimation tasks, but most of the previous work focuses on the analysis of the pelvis and the skull. More importantly, the humerus, which can be useful in some situations due to its resistance, has never been used in the development of an automatic sex estimation method. Therefore, this paper addresses the use of machine learning techniques to the task of image classification, focusing on the use of images of the distal epiphysis of the humerus to classify whether it belongs to a male or female individual. To address this, we have used a set of humerus photographs of 417 adult individuals of Mediterranean origin to validate and compare different approaches, using both deep learning and traditional feature extraction techniques. Our best model obtains an accuracy of 91.03% in test, correctly estimating the sex of 92.68% of the males and 89.19% of the females. These results are superior to the ones obtained by the state of the art and by a human expert, who has achieved an accuracy of 83.33% using a state-ofthe- art method on the same data. In addition, the visualization of activation maps allows us to confirm not only that the neural network observes the sexual dimorphisms that have been proposed by the forensic anthropology literature, but also that it has been capable of finding a new region of interest.European Commission FORAGE (B-TIC-456-UGR20

Automatic evolutionary medical image segmentation using deformable models

International audienceThis paper describes a hybrid level set approach to medical image segmentation. The method combines region-and edge-based information with the prior shape knowledge introduced using deformable registration. A parameter tuning mechanism, based on Genetic Algorithms, provides the ability to automatically adapt the level set to different segmentation tasks. Provided with a set of examples, the GA learns the correct weights for each image feature used in the segmentation. The algorithm has been tested over four different medical datasets across three image modalities. Our approach has shown significantly more accurate results in comparison with six state-of-the-art segmentation methods. The contributions of both the image registration and the parameter learning steps to the overall performance of the method have also been analyzed

Bayesian neural networks increasingly sparsify their units with depth

We investigate deep Bayesian neural networks with Gaussian priors on the weights and ReLU-like nonlinearities, shedding light on novel sparsity-inducing mechanisms at the level of the units of the network, both pre-and post-nonlinearities. The main thrust of the paper is to establish that the units prior distribution becomes increasingly heavy-tailed with depth. We show that first layer units are Gaussian, second layer units are sub-Exponential, and we introduce sub-Weibull distributions to characterize the deeper layers units. Bayesian neural networks with Gaussian priors are well known to induce the weight decay penalty on the weights. In contrast, our result indicates a more elaborate regularization scheme at the level of the units, ranging from convex penalties for the first two layers-weight decay for the first and Lasso for the second to non convex penalties for deeper layers. Thus, despite weight decay does not allow for the weights to be set exactly to zero, sparse solutions tend to be selected for the units from the second layer onward. This result provides new theoretical insight on deep Bayesian neural networks, underpinning their natural shrinkage properties and practical potential

Bayesian neural networks become heavier-tailed with depth

International audienceWe investigate deep Bayesian neural networks with Gaussian priors on the weights and ReLU-like nonlinearities, shedding light on novel distribution properties at the level of the neural network units. The main thrust of the paper is to establish that the prior distribution induced on the units before and after activation becomes increasingly heavier-tailed with depth. We show that first layer units are Gaussian, second layer units are sub-Exponential, and we introduce sub-Weibull distributions to characterize the deeper layers units. This result provides new theoretical insight on deep Bayesian neural networks, underpinning their practical potential. The workshop paper is based on the original paper Vladimirova et al. (2018)